DOCUMENT RESUME 



ED 222 532 TM 820 683 

AUTHOR Ree, Malcolm James; And Others 

TITLE Calibration of Armed Services Vocational Aptitude 

♦ Battery Forms 8, 9, and 10. 

INSTITUTION Air Force^ Human Resources Lab., Brooks AFB, Texas. 

REPORT NO AfHrL-J:R^81-49 
PUB DATE Feb^^ 
NOTE 19p. 

EDRS PRICE MFOl/PCOl Plus Postage. 

DESCRIPTORS *Aptitude Tests; *Armed Forces; Enlisted Personnel; 

*Equated Scores; Test Reliability 
IDENTIFIERS *Armed Forces Qualification Test; *Armed Services 

Vocational Aptitude Battery; Calibration 

ABSTRACT 

A calibration of the Armed Forces Qualification Test 
(AFQT) composite of the Armed Services Vocational Aptitude Battery 
(ASVAB) Forms 8a, 8b, 9a, 9b, 10a, and 10b to the metric of the AFQT 
Form 7a (AFQT-7a) and a comparison of these outcomes to the 
operational calibration tables implemented 1 October 1980 are 
presented. A sample of applicants for military enlistment was 
administered one form of ASVAB and the AFQT-7a in counterbalanced 
order. For analytic purposes, an edited sample (15,115 males) was 
separated into 6 samples based on the 6 forms of ASVAB administered. 
Data were collected at 20 geographically dispersed Armed Forces 
Examining and Entrance Stations on the 6 forms of ASVAB and the 
AFQT-7a. Each of the six samples was edited and scored, and 
descriptive statistics were computed. The root-mean-square and 
average absolute deviation measures, investigating the similarity of 
the equated scores across the forms, showed only small differences 
among the operational table and tables developed during this study. 
Forms 8, 9, and 10 of ASVAB were found to be parallel when equated to 
AFQT-7a, and a single conversion table was deemed appropriate for 
operational enlistment processing. (Author/PN) 



*************************************************************** 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 
*********************************************************************** 



ERLC 




H 
U 
M 
A 
N 

R 
E 
S 
0 

U 
R 

C 
E 
S 



PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC)." 



U.S. OEl^ARTMENT OF EDUCATION 

NATIONAL INSTITUTE OF EDUCATION 

EDUCATIONAL RESOURCES INFORMATION 
CENTER (ERIC) 
This d(H:umpnt hds been reproduced as 
rf'ctMvttd ffuni thf person or organization 
ofiginatinti it 

Minor chdn..}(»s hdvj? been made to improve 
roprodu. ti(in quality 

• Points of view nf opinions stattJd m thisdocu 
mnnt dn tiot nuc (!ssafiiv ruprcsent official NIE 
positinr iU ptiiii V 



CALIBRATION OF ARMED SERVICES VOCATIONAL 
APTITUDE BATTERY FORMS 8, 9, AND 10 

By 

Maloolm James Roe 
John J. Mathews 
Cecil J. MuIIins 
Randy H. Massey, Capt, USAF 

MANPOWER AND PERSONNEL DIVISION 
Brooks Air Force Base, Texas 78235 



February 1982 
Interim Report for Period October 1980 - July 1981 



Approved for public release: distribution unlimited. 



LABORATORY 



NOTICK 



When GovernmeiU drawings, sperificalions, or olhor data are used for any purpose other than in 
connection with a definitely Government-related procurement, the United States Government incurs 
no responsibility or any obligation whatsoever. The fact that the Government may have formulated or 
in any way supplied the said drawings, specifications, or other data, is not to be regarded by 
implication, or otherwise in any manner construed, as licensing the holder, or any other person or 
corporation; or as conveying any rights or permission to manufacture, use, or sell any patented 
invention that may in any way be related thereto. 

The Public Affairs Office has reviewed this report, and it is releasable to the National Technical 
Information St^rvice, where it will be available to the general public, including foreign nationals. 

This report has been reviewed and is approved for publication. 

NANCY GUINN, Technical Director 
Manpower and Personnel Division 

RONALD W. TERRY. Colonel, USAF 
Com mander 



3 



SECURITY CLASSIFICATION OP THIS P A E rWhrn Datu Eiit^'n-it^ 


REPORT DOCUMENTATION PAGE 


KKAI) INSTRUCTIONS 
BKFOKK rOMF'UETINCi KORM 


1 REPORT NUMBER j^. GOVT ACCESSION NO. 

\i iiRi .rK-r.i-i'j j 


3 RECIPIENT'S CATALOG NUMBER 


4 Title r«ru/ Subtmo) 

( \M!iU \ rin\ or M M i:^ \(M\Tl()N M 


5 TYPE OF REPORT A PE.RIO'J ■' Ov^'EREP 

[nlctiiii 

Or i,)l).T - Jnls VnW 


6 PERFORMING 0"^G. REPORT NUMHER 


7 Aw'^HORs. 


8 CONTRACT OR GRANT NUMBER(S. 


PtRP'>RMIN OROAnI Z ATiON NAME AND ADDRESS 
Mjnjmwcr ,111(1 I'r'r-nliiirl |)isi«.ti)n 
\«r 1 (in f liiiriMii R«"«i)n ir 1 .liniialnr s 
UrHuk- Vir 1 III ! I' j>.i-r. 1 i'\4i«- . i ) 


PROGRAM ELEMENT, PROJEC"^. T Af.K 
AREA d WORK LIN IT NUMBERS 


M. CONTROLLING OFFICE NAME AND ADDRESS 

l(n \ir l iin i' Minn, in Kr«.iMMr r*«. 1 .1 hor.i t in\ f M >t ' 
i>tMMk' \ir liifir U,i««i>. Ir\.j«. TH^ii 


12. REPORT DATE 

Kchniarv m^l 


13, NUMBER OF PAGES 


14 MON!'^DRlN3 A3r:N"Y NAME ^ ADDRESSf'j/ difforfnt (mm ' nntmllrnd Olft.r) 


15. SECURITY CLASS, (nl thi^ rt'P'^rt 

[ Ml lil^^tl mm! 

~Sa ~ d'e CU A SSI n'c AT^ION DOWNGRADING 
SCHEDULE 



16 DI STRi 9' 1 Ti ON STATEMENT ''nf rhin KcpnrfJ 



\|)[iri»\ im1 lor [iIiIiIm irlrM^c; rli-lnblinuM iinlmiilr-f| 



17 DrSTRl3UT!ON STATEMENT fhe^ abfitrart entfyml :n Bl^rk 20^ it ^tiffvront itnm Rvpnrt) 



18. SUPPL EMEN TARY NOTES 



19 KEY WORDS (Continue on rt^vors 


> ^/f/e if nece.nsary and identify by blovk number) 


illnlifs h'olin^ 


rursc ^irioolhiu*: 


\i or 




.ijihlirrlr' l('<.l> 


{'(juifX'n (Mililr 




|H)I\ nr)rii i.tl r<'j,n"f»«.i()ii 


( iiiilirmint) 





20 ABSTRACT n'outtnut^ on rover.io side if necossnry mnd identify by block ntjmbar) 



I'lu' nlijc( li\c to ( iililjrah* llw Vriiu'*! l on-c^ (^)iialirirnlion 'Vol ( M'(/i') coijijk)^!!!' oI liw Vr iiicd Scrsiic- 
XcHiiiiori.il \|)lilu<l.' fiiitirrs ( \S\ \H) 1 orrii^ Ha. »l>. <)li. lOa.iuul I Oh to I he iin-l rir of I hr MOT Form Ta ( MO'i'- 
7.i) and In fnin[)ar(* llw-r oijlroiru-- lo ihr r)|HTali()iial ralihralijui laMf- i in plfnir n Ird 1 Oclohcr PW). \ >aiiij)l«' of 
appiirani • Inr inililars cnli-^lnwiit .ul imiii>»h'n'd our form of \S\ Mi and llu' \r(,)T-Ta irj con nirrhalanrrd order, 
I nun ihiH \,\iirr\ Hjniph- o! 22. IIH). .i ■ iiiiilc^ onls ^.inifjlr ol I 'i. I I ,"> sva- d»'s clojX'd llironj^li data cdilint: trcluinjuc^ 
dt'siifrird to f'\( Inde Imialr^ and l a^i'- u illi irn (miplclc or nnn>al)li' data. I'Or anal \ ll( pn rpox'^. I lii^ rdilcd ^amplr wa^ 
H^paralrd iiiio ^ix HanipM'^ h.i-cd <m llw lorin^ of \S\ \H admini^lcrcd. Dala were ( ollrclcd at i^M-oj^n-afihit ;ill\ 
diHp('r-<'d VriiKM! f on»'- |-.\amiiim^' and (jilrainc .^laliniw ( M"|'!I'!S) on tlir -i\ lorm- nl \S\ and llw VM/T-Ta. 



I m'la--ifi('d 

SECURITY CLASSIFICATION OF THIS PAGE (When Dmtm Enimfd) 



DD 1 JAN 73 1473 EDITION OF 1 NOV 65 IS OBSOLETE 



ERIC 



SECURITY CLASSIFICATION OF THIS PACE(Wh9n Dmtm Entmrmd) 



IhMii 21) ((iodtiimcd): 

Knell {)\ llu' >i\ inal('> oiih samples N\a>^ edit*'*! and >r()r»'d. and dt'>{'ri plivc stali^tio wrrr cotnputcd. Percentile^ lor 
bolli (he \S\ \\\ and llw \f'(/r-7a Nsere equated and smoothed l)\ a (XjUnomial re^re»ion j)roeednre. Kaeh sample 
>\as >plil in hall, and the (Mjuatin^ and smoothing were repeated on each half sample. Since re>Mlt> were eon>lst(Mit 
ainon^' the lar^^e >ample and the two half sample^, tlic) were accef)ted. Ifi order to in\ estimate the similarit\ of the 
equaled scores aero>> the form^. root-mean-scpiare (HMS) and averaf^e al)solute deviation (AM)) nH'jj>ures were 
computed between the varion> ecfuatin^ tables. A comparison of the forms found them to l)e efjuiv^dent when they 
werf' e(pial<'d to M-OT-Ta. The RMS and AAl) measure- showed oid\ small differem'e> amon^ the operational table 
and table> dcN eloped during this stud>. !*orni> 8. and 10 of AS\ AB were foumi to be (Kirallel w lien e(pialed !o 
Vf'OT-Ta. and a >iii^le coinersion table was deemed a[)[)ropriate for operational erdistment proce>>in^. 



I nelassified 



SECURITY CL ASSIFICATIOH OF P ^GE(Whmn Dmtm Enfrmd) 



PREFACE 



This study was completed under the auspices of Personnel Qualification Systems wliich is part of a 
larger effort in Force Acquisition and Distribution. It was subsunu^d under project 77191801. "Maintenan<'e 
and Improvement of Enlisted Selection and Classification Tests'* and executed as part of (lie responsibility of 
the Air Force Human Resources Laboratory (AFHRL) as lead laboratory und(T the executlvr agent (Air 
Force) for Armed Services Vocational Aptitude Battery research and development. 

An effort such as this, although under the direction of an individual, can be accomplished only through 
a team effort. The authors wish to express their appreciation to Roy (^hollman, Jame.s Earles. Jcnn\ 
Hodge, and .AlC (herald Yates. A debt of gratitude is owed to Doris Black, who served to condense arul 
translate the analytic requests into operational procedure for the Technical Services Division. Henry (lark 
wrought minor tnagic by convincing the c()mf)uter to produce analys(\s prior to estal)lislied due dates. 

The authors also wish to express their appreciation to Jacob! na Skinner and the otlier meml)crs of 
F^iblicntion Review Panel 2 for helpful comments on an earlier draft of this mannserl{)(. 



ERIC 



TABLE OF CONTENTS 



1. Introduction 5 

(lalibration of Tests 5 

il. Method - 6 

The Tests ^ 

Administration of Tests lo Subjects 7 

Data Editing 8 

Sample 8 

Equipercentile Equating and Calibrating 8 

Table Generation 9 

III. Results and Discussion 10 

Data Editing 10 

Analysis Samples 11 

Descriptive Statistics 12 

Equating 12 

Tables for the AFQT Forms 12 

IV. (Conclusions 1^ 

References 1^ 

LIST OF ILLUSTRATIONS 

Figurt* P'M?^ 

1 Scatter Plot of Arithmetic Rea.soning and Numerical Operations Test Scores 11 

LIST OF TABLES 

Table P^g*- 

1 Name and Number of Items for Power and Speeded ASVAB Subtests in Forms 8, 9. and 10 6 

2 AFEES Sites and Target Sample at Sites 7 

3 Example of Smoothing by Polynomial ^ 

I Nurnber of Subjects Flagged by Key Verification by Test Form 10 

f) Example (Cases from Key Verification Procedures 10 

6 Number of Subjects by ASVAB Form 12 

7 Descriptive Statistics for ASVAB 8, 9, and 10 and AFQT-7a 12 

8 Conversion Tables for Each Form 13 

9 Deviation Measures Comparing Use of One Versus Six Conversion Tables 1^ 

U) Classification by Mental Category Based on One Versus Six Tables 15 

1 1 Deviation of Percentile Scores across Category Lines 16 



ERIC 



7 



CALIBRATION OF ARMED SERVICES VOCATIONAL APTITUDE 
BATTERY FORMS 8, 9, AND 10 



I. INTRODUCTION 

The measurement of human characteristics has been a necessary part of selection and claisification for mili- 
tary occupations for over 60 years. Like measurement of physical characteristics, such as length, weight, or density, 
no natural units of measure exist for psychological characteristics; rather, artificial units are established by consen- 
sus. One of the most frequently used units of measurement for human characteristics is the percentile equivalent. 
The percentile is reported in reference to some standard population or group. Ability tests used for military selec- 
tion and classification are usually referenced to the 1944 mobilization base, and 'iiis is usually accomplished by 
equating new tests to old tests. Equating is the conversion of score units of one test to the score units of another test. 
The current study describes the referencing of Forms 841, 8b, 9a, 9b, 10a, and lOb of the Armed Services Voca- 
tional Aptitude Battery (ASVAB) to the mobilization base metric, through the use of an anchor test. 

There are two important reasons why current tests are equated to past tests. The first is to enable the testing 
agency to report oa the relative distribution of scores on a year-to-year basis in a common metric. For example, the 
various military services I ke to be able to compare current accessions to past accessions on the same scale. The 
second reason is to provide a consistent meaning for cutting scores for selection and classification tests. In theory, a 
score for the new test at the 80th percentile can be said to be equivalent to a score at the 80th percentile on the past 
tests, and this equivalence becomes the definition of consistency. 

When several forms of a test are to be operational simulunec'isly, it is an advanUge if they are parallel, which 
allows the use of a single equating Uble. Gulliksen (l950) offers a definition of parallel tests which includes same- 
ness of factor structure, equality of means, equality of variances, and equality of non-zero correlations with an 
external criterion. It also seems reasonable to include equivalence of skew ana kurtosis (Ree, 1977), the thittl and 
fourth moments of the distribution, although little research exists in the ar^a. 

Parallel tests may be constructed by assigning items randomly to forms. This method is usually called "Ran- 
domly Parallel Forms." Or items may be matched on difficulty and/or discrimination, stratified, and then assigned 
randomly to one of a set of multiple forms. This procedure is called "Stratified Parallel." Analytic method.^ of con- 
structing parallel forms also exist (Ree, 1976), but they tend to be intensive of computer time. 

Using the Stratified Parallel method. Forms 8, 9, and 10 of the ASVAB were constructed to be parallel in 
terms of raw scores so that a single table might be used to convert raw scoreB on any of the six forms to percentile 
equivalents. The objective of this study was to determine if a single table were appropriate. 

Calibration of Tests 

Because two or more forms of a test can never be made precisely equivalent in range and level, it is necessary 
to render the forms interchangeable by equating. The equating procedure may be defined (Flanagan, 1951; 
Angoff, 1971) as converting the scoring units of one test to the scoring units of another. 

In general, two procedures have been in common use: linear and equipercentile equating. Linear equating 
requires that equivalent Z-score transformations of the two testd represent the same cumulative proportion. Said 
differently, the shapes of score distributions should differ only trivially. Equipercentile equating, on the other 
hand, makes no such assumption of Z-scoie equivalence. The linear method offers the advantage of dealing with 
analytic sUtistics (means, sUndard deviations, etc.) which are verifiable. Equipercentile equating if preferable 
when the distributions differ and is often offered as the definition of equating (Jaeger, 1981). It should be noted 
that the linear and equipercentile approach coincide when both the distributions to which they are applied have 
the same shape. 



Angoff (1971) U8e8 thf* t^^rm ''calibration" to describe the equating of tests of differing abilities. For example, 
the equating of a test of Word Knowledge to a test of Reading would be called ''calibration." Therefore, it is 
appropriate to say that military selection and classification tests have bee\i calibrated rather than equated. Angoff is 
somewhat critical of the calibration technique because a problem arises from the nature of calibration. It is 
repeatedly sUted in the literature (Angoff, 1971; Flanagan, 1951; Jaeger, 1981) that calibrating does not lead to 
sample-unique solutions, as does equating, although empirical evidence is not offered. The non-uniquent»ss of the 
solution makes difficult the interpretation of several calibrations of the same test, or parallel forms of the test. 
MihUry selection and classification tests have frequently been calibrated, rather than equated. Form 8a of the 
ASVAB was linked via caHbration to an anchor test using several differing subject groups ranging from high school 
students to new military recruits. The effects of calibrating, as opposed to equating, require further study in order 
to understand fully the consequences of the technique. 

Three previous studies (Boldl, 1980; Maier & Grafton, 1981; Sims & Truss, 1980) were conducted which 
calibrated Form 8a to Armed Forces Qualification Test Form 7a (AFQT-7a). Because ASVAB Forms 8, 9, and 10 
were constructed to be parallel by the method described previously as "Stratified Parallel Forms," it was reasoned 
that calibrating one form was tantamount to calibrating all forms. That is, because calibration sets raw scores of the 
calibrated test equivalent to raw scores on an anchor or target test, and because the raw scores of the six forms were 
constructed to be equivalent, then any one form may be calibrated, and the results should then be applicable to all 
the other forms. The crucial requirement is that the forms be parallel. If they are not. separate calibrations are 
required. The present study seeks to verify the results of the earlier calibration studies which produced the tables 
implemented 1 October 1980. These are referred to as the operational tables. 

In order to determine if the assumptions underlying the procedures for calibrating ASVAB-8a and thereby 
Forms 8b, 9a, 9b, 10a, and 10b were accepUble, an Initial Operational Test and Evaluation (lOT&E) was 
underUken. The lOT&E was begun shortly after the test was put into operation for selection and classification of 
candidates for military enlistment. 



II. METHOD 



The Tests 

Forms 8, 9, and 10 of the ASVAB are multiple aptitude batteries comprised of 10 subtests. Eight of the 
subtests are power subtests, while two are speeded subtests. Table 1 shows the name, the number of items, and 
whether the subtest is power or speeded. These forms differ from the previous ASVAB forms by the inclusion of 
Paragraph Comprehension (PC) and Coding Speed (CS) subtests, by the combination of Automotive Information 
and Shop Information into a single subtest (AS), and by the deletion of subtests measuring Space Perception, 
Attention to DeUil, and General Information. The overall administration time for any of the forms is about 180 
minutes, and in operation, the test is answered on a machine scannable answer sheet. 



Table L Name and Number of Items for Power and Speeded ASVAB 
Subtests in Forms 8, 9, and 10 



Name 



General Science (GS) 
Word Knowledge (WK) 
Arithmetic Reasoning (AR) 
Paragraph Comprehension (PC) 
Numerical Operations (NO) 
Coding Speed (CS) 
Auto-Shop Information (AS) 
Mathematics Knowledge (MK) 
Mechanical Comprehension (MC) 
Electronics Information (EI) 



r of Itemt 


Power/Speed 


25 


Power 


35 


Power 


30 


Power 


15 


Power 


50 


Speed 


84 


Speed 


25 


Power 


25 


Power 


25 


Power 


20 


Power 
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The Armed Forres Qualifiealion Test (AFQT) composite is used for mililary enlislmenl Cjualilu aliori and is 
comprised of PC, Word Knowledge (WK), Arithmetic Reasoning (AR), and Numerical Operations (NO) sul)tests. 
All subtests are unit weighted except for NO, which is weighted by one-half. 

The AFQT-7a served as the anchor test. This test was previously used for enlistment qualification but has 
been inactive for several years. It was chosen as the anchor test because its content is close to that of the test used in 
the 1 W mobilization base development testing. It is not believed to be compromised, and an earlier form (Form 3) 
of the .\SV.\B was calibrated against it. 

The AFQT-7a has 100 items evenly distributed in the ability areas of WK, AR, Boxes (B), and Tool 
Knowledge (TK). The first two, WK and \R. are similar to the like-named subtests in the curretit AFQT portion of 
the .ASVAB. The latter two, B and TK. are not found in the current AFQT portions of the ASVAB. It is the disparity 
in the ability areas measured which leads to labeling the equating effort a ^^calibration^' and which leads to the 
problem of non-unique solutions. 

Administration of Tests to Subjects 

A sample of subjects was drawn to provide for equal geographical representation. Data collection took place in 
20 Armed Forces Examining and Entrance Stations (AFEESs). Tabl(^ 2 shows the locations of the AFEESs and the 
number of subjects at each. Each subject took the AFQT-7a and one form of the ASVAB, which was used for 
qualification for military enlistment. The AFQT-7a was administered on a separate answer sheet. The ASVAB and 
.\FQT-7a tests were administered in counterbalanced order by reversing order of their administration each day 
from that employed the previous day. Tests were also administered at locations affiliated with the AFEES, called 
Mobile Examining Team (MET) sites and Office of Personnel .Management (OPM) sites. 



Table 2. AFEES Sites and Sample at Sites' 



AFEES 


Subjects 


Chicago 


1,500 


Cleveland 


1,300 


Atlanta 


800 


Baltimore 


1,600 


Boston 


1,300 


Jacksonville 


1,400 


Los Angeles 


2,600 


Montgomery 


900 


Newark 


1,400 


Philadelphia 


1,400 


Richmond 


1,200 


St. Louis 


1,400 


Spokane 


500 


Denver 


600 


Houston 


600 


Phoenix 


500 


Portland 


400 


San Diego 


600 


Minneapolis 


1,200 


Omaha 


1,200 


lot^l 


22,400 



^Silr« included AFEES, MET, and OPM Iwalions for test administralion. 
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Data Kditiiig 



Ml answer slierls wvrv vlsunlK inspccUMi for coinplrtcnt^ss of iiii'orrTintion and stra\ ruark>, Tfic \S\ AB uses a 
tfirrt'-part ans>vrr sfu^ct whicfi l> optical!) Ncaimal)!^ and has prccodcd nundicrs on each shrrt to keep the triplet set 
inlaet during operational seaiunng. There is also an opticalU scannable social security account runnhvr (SSW) 
n^rid. 'I'fiese operational \S\ Afi answer sfieets which had heen scanned at AKKKS were then rescanned and the 
required triplet> of answer slieets were merged, The \FQT-7a answer sheets were also scanned and iner«ie>l with 
lh(» records of the ASVAB for each suhjeet. lieeausc ordy nudes were represented in tfie Orld War II (I'^U) 
iiU)l)ili/ation has(», female suhjecls were deleted iVoni the original saniple to leave a ""males onl\** sample of 
a[>plicant>. 

Tfiree other editing j;)rocedures were emplo\ed, First, to deterinifU' if tlie correct form of the test (8a to lOh) 
was speeifieci on the answer sheet, a cfieek was perfornu'd by scoring tfie first four items in the ^(). VS. and W K 
sul)te>ls. Twelve items in all were scored, The NO aiui (IS are speeded subtests, and tlie Vi K sul)test has the easiest 
items first. It was reasoiU'd that an\ examiru'c's score of 6 or less was suspect and should be examined further. This 
was aceomplisfi(»d bv a[)plving each of the six foriTi-specific scoring keys for tliese 12 items to tlie answer sheet and 
comparing the magnitude of tlu' scores from tfie various key sets, I'or example, if tfie suf)ject coded "Form 8a on 
tlie answer sheet and obtained a score of 2 from the Form 8a key, but wfien scored on the Form lOb key obtained a 
score of 1 1, tfien the cTitire lest was scored using tfie lOb scoring key. If. on the otfier fiand. low scores were found 
for alt forms, then the ke\ for the form indicated by tfu* examinee was retained. 

The seeonci (lata editing procedure was designed to see if differences existed anu)rig types of testing sites: 
\FKFS, MKT, and OPM, This was accomplished by inspecting tfie nu'an and standard deviation of the absolute 
differences, by type of test site, betweei. the scores on the Af'QT-7a and the AFQT portion of the six forms of the 
ASVAB. Systematic deviance in a type of testing site would iiuiieate tluit data from that kind of site should be 
discarded. 

The third and final cheek was to investigate the bivariate scatter plots and standardized residuals devolved 
from regressing scores for each ASVAB-AFQT on scores on AFQT-Ta. scores on each AR on Math Knowl('<ig<' 
(MK), ,'.tid scores on each NO on ('S, These three sets of variabl(*s allow investigation of consistency of responding 
between the first and secomi halves of the ASVAB for botfi pow(*r and speeded tests as well as between a test 
acUially used for military enlistment qualification (ASVAfi) aiul a test (AFQT) given for equating purposes only, 
Kach [)air of variables is highly correlated, KxamirU'es with standardized residuals outside of the range of ± 2,50 
were icientified for further scrutiny, Tfiey were located on the appropriate scatter plot and were deleted il it was 
reasonably clear from visual inspectiofi that they represented true outliers by being substantially away fron\ the 
bulk of the scatter. 

Sample 

Froin the original sample collected at tfie AFFES, MFT, and OPM sites, females and those w ho failed the data 
editing were remove(i. Six mah'-ofily sairiples were created based on tfie form of ASVAIi administered. Random 
half-siimples were selected wi'vbin each of the six male-only samples created for F{)rms 8a ifirough lOb, These fialf- 
samples were established in order to cross-validate results and to investigate consistency of various estimates made 
in the equating process. 

Fqui percentile Equating and Calibrating 

It is appropriate to specify that Forms 8, 9, and 10 of ASVAB were calibrated using AFQT-7a as a standard. 
The plan identified as "Design IF* by Angoff (1971) was used for cacfi pair of composites to be calif)rated. 

Test calibration was accomplished using raw scores on th(» ASVAB-AFQT and o!i the AFQT-7a as a starting 
point. For each raw score distribution of ASVAB-AFQT and AFQT-7a, sample dependent percentiles from 1 to 9<) 
were computed in unit interval.s. Tfiis is essenliaHy a raw score to raw score procedure. F'revious eqnating.s using 
ASVAB-AFQT raw score to AFQT-7a percentile equivalents only, rather than ASVAB-AFQT raw score to AFQT-7a 
raw score were deeme(i insufficient, as information was lost wfien raw score point intervals were collapsed. The raw 
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score to raw score procedure was used because it is more widely accepted and more efficient. After the raw score 
equivalents were established, it was necessary to smooth the resulting line. This smoothing was accomplished by 
using the analytic procedure of polynomial regressions up to the third order. The fit of the regression was used to 
determine the best curve. 

The creation of half-samples was especially useful in determining the relative stability of the quadratic and 
cubic regression weights. Each smoothing was accomplished three times, and the weights were retained only if they 
remained relatively constant. The cases in which higher order, weights did not remain constant were smoothed by 
the first order polynomial, as it always remained constant. Table 3 provides an instructive example using invented 
data. The samples 1 through 3 on the left show instances where the weights (W;) are stable and thus are acceptable 
to smooth the equating line. The fourth, fifth, and sixth samples show an instability of weights due to capitalization 
on chance fluctuation, which causes the high order polynomials to be rejected. Note how the values in the columns 
marked ''W2" and ''W3" fluctuate in these later samples but not in samples 1 through 3. This kind of instability of 
weights should be the basis for rejection of the polynomial. Note also how the standard error of estimate (SEE) 
decreases substantially as the higher order terms are entered in samples 1 to 3, hut not in samples 4 through 6. This 
consistency and reduction of SEE is indicative of a better fit. Three additional points are worthy of note. First, the 
is observed to change only in the trivial third decimal place, and little emphasis should be placed on it. 
Secondly, the standard error of estimate is appropriate for determining fit. P'inally, care must be exercised not to 
interpret the R and R"^ as correlations between raw scores for subjects. These indexes reflect the covariation of the 
equated percentile points in a distribution and must be expected to be quite high. One advantage of this method of 
smoothing is that it is analytic and reproducible, thereby avoiding the myriad pitfalls of hand smoothing. 







Table 3. ExampU 


of Smoothing 


by Polynomial 




Sum pit' 


Type 
















Composite I 






I 


Full 


.9087 


1.0 




2.618 


I 






.981 


.056 


1.072 


I 






.970 


.019 .051 


.671 


2 


flalf 




i .050 




3.012 


2 




.9999 


.980 


.060 


i.iOI 


2 




.9999 


.970 


.051 .050 


.801 


3 


Flaif 


.9989 


1.055 




3.000 






.9999 


.980 


.058 


1.300 


3 




.9999 


.971 


.049 .052 


.790 








Composite 2 






i 


Full 


.9999 


1.061 




2.710 


i 




.9999 


.982 


.311 


2.600 


1 




.9999 


.961 


.032 .202 


1.930 


5 


Half 


.9981 


1.059 




2.950 


5 




.9999 


.931 


.103 


2.710 


5 




.9999 


.929 


.009 .001 


2.070 


6 


fialf 


.9992 


1.072 




2.870 


6 




.9999 


.901 


.081 


2.650 


6 




.9999 


.918 


.050 .too 


1.800 



Table Generation 



The ultimate goal of this effort is to produce tables for each ASVAB AFQT composite from Forms 8a through 
lOh and to determine if a single Ubie for each composite is applicable across the set of six forms. The tables were 



32 



»j»*TU'rahMi li\ pii kin^ rfic af)(>r()[)ricUf' sninofli f ur\ »' form and t*valualing it al t'ach raw score point tor llu' ran^e ot 
ilij* VI nl romposiu.. Ihis six rquatinij tables, one lor each \SV AB iorni. An average table v,ah created 

ironi ihesr si\. >e\»*ral (le\iali()n indexes were (oniputed make eoiuparisons among these tables and the 
opi-ratnmai table, l lu sc indexes were the r()ot-nu^an-s(|nare (RMS) deviation and average absolute deviation 
* V M)) VibiitionaiU . ihr ^^imiiantN beiwern classilu atioM into mental categories (see (rrunzke, Ouinn. & Siauller, 
I'^TtU h\ fill- operalinnal table and the si\ lurni-spct ilii tables was investigated by computing a iwo-wav fre(|uenc\ 
'aid.' »d ria-iliraUo!! 



in HVM I I s KM) DIM I NSION 

l)ata Fditiut: 

llu- . h 'rk U> %I»'!frnun»* li tht- n>rrert lorm (Ha through iOb) wa> coded produced 127 subjects rec^uiring 
I I able 1- '-hiiws t|u* number oi ra^^es^ b\ lorni. winch were idenlilied lor veriticalion. For all the lorms, '^2 

I - wf V' dt i''f» d. ^1 -had lurni rhangt s, and were hdt unchanged. 



laliif t. Number of Subjects Flagged bv Key Verilieation 
Test Form 



Sub] ects 





I Ola! 


Not Ke> fla^jurcd 


Ke) Flagged 


Ha 




2:)(>i 




Hh 


2:)2M 


2t:7 


:)2 




2t)2,^> 


2^> i^) 


7t) 




2:)2: 




()n 




2.^)10 


2 12^) 


HI 




2U8 


2:^(>^) 





H\ svav of ^^xamph . fo-ir ( a>es disphued in 1 able f) are instructive. Case 1 was changed to 8b because of the 
1..W sri.n on 8a . ninparcd to llie high s( ore on 8b. Case 2 wa,s deleted because having a one or zero on all scoring 
kev. uidirated the t xaininee was unlikeK to have been trying very hard. Case 3 was deleted because it was 
iniposMble tti d.-ieinijT.e whi( h test the e\aiuinee was administered, as the form coded on the answer sheet had the 
rr iA flu- six l as.' 1 Wcxs kept, despite liie low scores, since the score for Mie form coded wa.i the highest. 



low* 



Idhh J. F^xainple Cases from Kev Verification Procedures 



Scores for Forms 



rornH.wlrtl Bu Hl> Ob 10a 10b 



Ha [ in 2 1 3 

8b I 0 0 1 1 0 

•^a :i I 1 ^ 

Wb 1 i (y ^ 1 



1 hr srrond data editing procedure of investigating differences among lype.s of testing sites by comparison of 
absolute differences on \FyT-7a and ASVAB-AFQT revealed no systemalic differences. Consequently, all site 
l\pe^ W(Te deemed appropriate for inc lusion in the study. 

The third and final check was to investigate the bivariale scatter plots and standardized residuals devolved 
from regressing scares for each ASV \H-AFQT on scores on AFQT-7a, scores on each AF{ on MK, and scores on each 
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NO on (IS. Examinees with standardized residuals outside of the range of + 2.50 were identified for further 
scrutiny. Each was located on an appropriate scatter plot, and tlie score was deleted if it was clear that the examinee 
represented a true outlier by being substantially away from the bulk of the scatter. It was observed, for example, 
that some examinees displayed high scores on AR but very low scores on MK. This is an illogical situation that 
might be accounted for by having obtained some answers for the AR subtest, which is in the qualification portion 
of the ASVAB, but not for MK. It might also be an indication of faltering motivation on the later MK test. In either 
case, the examinee should not be in the sample. Figure 1 shows this condition. The observations within the dotted 
boundaries were subject lo scrutiny and potential deletion. Only 132 subjects were removed during this procedure. 
The finals^^ple was comprised of 15,115 male subj^*ts. 



A 
R 
I 
T 
H 
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E 
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I 
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R 
E 
A 

0 
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r 
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36 



30 



24 



18 



12 



NUMERICAL OPERATIONS 



Figure L Scatter Plot of Arithmetic Reasoning and 
Numerical Operations Test Scores. 



Analysis Samples 

Table 6 displays the sample sizes for each of the six male-only samples. 
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Table 6. Number of Subjects by ASVAB Form 





Number of Subjects 


8a 


2,621 


81) 


2,5()() 


<)a 


2,587 


Oh 


2,500 


lOa 


2,181 


lOh 


2,117 



Descriptive Statistics 

Table 7 shows the descriptive statistics for each of the ASVAB subtests, the ASVAB-Af\)T, and AF()T-7a, 
can be seen, the means (\) differ relativeJv litthv as do the standard deviations W. Ciiniuiative freqne 
distributions of the scores are of the same general shape with few differences among them. 

Table 7. Descriptive Statistics for ASVAB 8, 9, and 10 
and AFQT-7a 



ASVAB Form Adrninisttrcd 



8a) nh 9it 9b lOa M)b 



Sub- 




























\ 




\ 


(T 


\ 


(T 


\ 




\ 


<j 


\ 




(;s 


15.2^) 




15.10 


l-/)2 


1 l-,61 


5.51 


1 |.,50 


5.51- 


1 |..6() 


5.00 


1 1.71 


5.15 


AR 


H). 1-7 


().7() 


17.13 


7.13 


16,02 


(),06 


17.28 


6,86 


1 7.93 


6.70 


17.00 


6.98 


WK 


2 k()l 


7,55 


23.H- 


7.5() 


23,53 


7,66 


23,72 


7,75 


22,09 


7.82 


23. 13 


7.60 


PC 


10,08 


3,38 


t).8l- 


3.31- 


9,27 


3. t8 


10,02 


3,28 


9.59 


3,77 


10.02 


3.17 


NO 


31,52 


10.17 


3 t.75 


lO.Of) 


3 l-,20 


io,r)8 


33,03 


10,M) 


35.03 


10.0 1 


3 1.58 


10.30 


cs 


H,2f) 


15,()t 


U.27 


15.23 


1-1,1-2 


l.'"),05 


U,7:) 


1 t,53 


1-2.3 1 


11,81 


12.08 


11. 12 


AS 


15,25 


5.82 


1.'").21 


5.76 


l.'"),77 


.'),77 


15.7V 


5,71 


15.77 


5,65 


15.83 


5.6() 


Mk 


ii,:52 


5,5 i 


11.1 1^ 


5. 1-3 


11.21 


5, U) 


11,20 


5,60 


12.33 


5,33 


12.35 


5.56 


MC 


1 1 11 


5. 1-3 


1 l-.l t 


5.1-1 


1 1,28 


.5,33 


1 1,32 


5,07 


I 1. 1-5 


5,25 


1 1.27 


5.20 


K\ 


1 1.50 


t,31 


11.16 


1.20 


11,01 


t,13 


12,05 


3,98 


12.06 


t.03 


11.75 


1.03 


VK 


:u,72 


10.15 


33.28 


10. M) 


32,80 


10,63 


33,73 


10,55 


32.58 


1 1,00 


33. 1() 


10.26 


\F(/r 


(»8.()0 


10,22 


()8.02 


10,7t) 


67,10 


10,88 


()8.22 


)0 78 


68.27 


10.85 


68.20 


lO.Ol 


QT-Ta 


51.77 


20.80 


5t.37 


20.0 1 


5l-,68 


21,02 


51,01 


21.05 


5 I..89 


20,77 


55. 10 


20.82 



Note. \I'\Vr-7a is de noted by QT-7a. 
Equating 

All of the AFQl' composites were calibrated iisin^ the AFQT-7a as the standard and were smoothed using 
polynomial regression with the constraint that the curve exhibit positive nionotonicity. This meant that the curve 
was not permitted to turn downward, whicfi woukl have provided two percentile points for a ,single raw score. 

Each composite was calibrated in a full sample and two randomly selected half samples. The smoothing was 
applied to each subsample independently, and all three were used to decide on the appropriate smoothing on the 
basis of consistency among the samples and reduced standard error of estimate. 

It is worth noting that the analytic procedure automatically provides a measure of fit, the standard error oi 
estimate. Hand smoothing, as used in previous equating studies of ASVAB-8a. does not provide such an index 
without hiborious computation. A goodness-of-fit of the equating curve for the previous studies was not assessed. 
This is one of the drawbacks to the nonanalytic method used previously. 



Tables for the AFQT Forms 

The tests were quite similar in frequency distribution and relationship to the calibration standard of AFQT- 
7a. This led to generally equivalent conversion tables for all six forms. Table 8 shows the conversions of each of the 
forms and the average correspondence of the six forms to the percentile standard or metric of AFQT<7a. 

o • 
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Table 8, Conversion Tables for Each P'orni 



Percentile for f*orni 



Rjiw 

Score 


Ba 


8h 


9« 


9h 


lOu 


lOb 


Overall 
Avg 


0-17 


1 


1 


1 


1 


I 


1 


1 


18 


1 


1 


•) 


1 


•) 


I 


1 


1<) 


1 


0 


•) 


0 


») 


1 




20 


1 


•) 


•) 


0 


•) 


•) 




21 




•) 


3 


0 


3 


•) 


3 


22 




3 


3 


3 


3 




3 


23 




3 


3 


3 


3 


3 


3 


2i 


3 


3 




4 


3 


3 


3 


2,") 


3 






i 


1- 


4 


1- 


26 




I 


5 


5 


4 


i 


4 


27 


i 


5 


5 


5 


5 


5 


5 


28 


5 


5 


() 


() 


5 


5 


5 


20 


5 


() 




6 


() 


(> 


(> 


30 


() 


6 




-T 


() 


(> 


() 


31 


() 




8 


1 








32 






8 


8 


8 




8 


33 




8 


<> 


8 


8 


8 


8 


34 


8 


8 


<) 


<) 


<) 






35 


8 


<) 


10 








<) 


3() 


<) 


10 


10 


11 


10 


10 


10 


37 


<) 


10 


11 


11 


10 


10 


10 


38 


10 


li 


I 1 


11 


11 


il 


11 


30 


11 


1 1 


12 


12 


11 


11 


11 


U) 


1 1 


12 


12 


1 2 


]2 


12 


J 2 


tl 


12 


12 


13 


13 


12 


12 


12 


42 


12 


13 


13 


13 


13 


13 


13 


43 


13 


13 


14 


14 


13 


14 


14 


44 


13 


14 


14 


14 


14 


li 


14 


45 


14 


14 


15 


15 


14 


15 


15 


My 


14 


15 


15 


15 


15 


15 


15 


47 


15 


15 


16 


16 


16 


16 


16 


48 


15 


16 


17 


16 


16 


16 


16 


49 


16 


16 


17 


17 


17 


17 


17 


50 


17 


17 


18 


17 


17 


17 


17 


51 


17 


17 


18 


18 


18 


18 


18 




18 


18 


10 


18 


18 


18 


18 



Pereen.ile for Korm 



Srore 


(la 


Bh 


9a 


9 b 


lOa 


lOh 


Ovemll' 

Avg 


53 


1 (> 

18 


10 




I V 




J V 


1 0 
1 V 


54 


10 


20 


J. 1 






on 


90 
ZU 


55 


Of \ 

20 


O 1 

21 


.) 


J 1 


J. 1 


0*) 


91 
Zl 


5() 


21 






•) •) 


•) •) 


O'J 


99 


57 


22 


23 


21 








9Q 


58 


23 


2 h 


25 


24 




J.) 


9 1 
Z4 


50 


24 


25 


26 


25 


2o 


li) 


9C 
Zo 


()0 


25 


26 


27 


2() 


2() 


Z 1 


9 A 
Z() 


()1 


2() 


27 


28 


2 ( 


27 


28 


0'7 


62 


28 


28 


20 


28 


20 




Of) 

zO 


()3 


20 


20 


30 


30 


30 


30 


30 


64 


30 


30 


32 


31 


31 


31 


31 


()5 


31 


31 


33 


32 


32 


32 


o o 

32 


()() 


32 


32 


34 


33 


33 


33 


33 


67 


33 


33 


3() 


31- 


31 


31 


34 


68 


31 


34 


38 


36 


36 


36 


36 


()0 


36 


3() 


40 


38 


38 


10 


o o 

38 


70 


38 


40 


42 


40 


10 


42 


40 


71 


to 


42 


44 


12 


12 


14 


A <\ 

42 


72 


i'l 


44 


4() 


11 


41 


1() 


14 


73 


. 44 


46 


18 


4() 


1() 


18 


46 


74 


18 


48 


to 


18 


18 


10 


A O 

48 


75 


to 


10 


50 


10 


40 


50 


40 


7() 


50 


51 


51 


Mf 




.J J 


«> 1 




51 


51 


52 


51 


51 


52 


51 


78 


52 


52 


54 


52 


52 


51 


r>3 


7<) 


51 


54 


56 


51 


54 


56 


55 


80 


56 


56 


58 


58 


56 


58 


57 


81 


58 


58 


60 


60 


58 


61 


59 


82 


60 


60 


62 


61 


60 


61 


61 


83 


61 


61 


63 


62 


61 


62 


62 


84 


62 


62 


65 


63 


62 


63 


63 


85 


63 


63 


67 


65 


63 


65 


64 


86 


65 


65 


70 


67 


65 


67 


67 


87 


70 


70 


72 


70 


70 


70 


70 


88 


72 


72 


74 


72 


72 


72 


72 



Table 8. (Continued) 



Raw 




PcrctMitile 


for K 


f»rm 




Overall** 
Avg 


Raw 
Score 






ereeiitile 


f(»r F 


>rin 




Overall" 

Avg 


8u 


8b 




9b 


lOa 


!0b 


Ra 


8h 


9a 


9h 


lOu 


]0b 


H<) 


11 


7^ 


76 


71 


71 


71 


71 


<)8 


88 


88 


89 


8<) 


88 


88 


88 


<X) 


7() 


76 


78 


76 


76 


76 


76 


<)<) 


<)() 


89 


90 


90 


89 


89 


89 


<)l 


78 


78 


80 


78 


78 


78 


78 


100 


<)1 


90 


92 


91 


90 


90 


91 


<)2 


80 


80 


81 


80 


80 


80 


80 


101 


<)2 


91 


93 


92 


91 


91 


92 


03 


81 


81 


82 


81 


81 


81 


81 


102 


93 


92 


91 


93 


92 


92 


93 


n 


82 


82 


83 


82 


82 


82 


82 


103 


91 


93 


95 


91 


<)3 


93 


94 




83 


83 


85 


83 


83 


83 


83 


101 


95 


91 


96 


95 


91 


91 


95 


<)() 


85 


85 


87 


87 


85 


85 


86 


105 


<)6 


96 


97 


96 


95 


95 


96 


<)7 


87 


88 


88 


88 


87 


87 


87 



















''Overall inrraji** hdsr<\ on coiiversiofi values pri(»r lo roiindiup to iiUegers. 
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In order to determine if the ASVAB conversion tables truly differ, measures of deviation of subject percentile 
scores were computed using the operational, average, and form-specific table. These measures were RMS and AAD 
between pairs of interest. Table 9 shows the RMS and AAD for the AFQT. Although there are some differences 
among forms, the magnitudes of the differences are quite small. This is quite consistent with the two previous 
analyses and reinforces a picture of relatively small differ^^nces. 

Table 9. Deviation Measures Comparing l^se fif One Versus 
Six Conversion Tables 



ASVAB AFQT (:om|>osUes 8a thru lOb 



Test Form 



fipHnso 



Pooled 8« 8b 9a 9b 10.. 10b 



AAD 

{) vs. P .92 .79 .83 1.31 .(»8 .<)7 .'>8 

0 V.S. A .56 .88 .17 .65 .16 .25 .53 

A vs. P .65 .62 M .65 .67 .()5 .65 

RMS 

() vs P 1.25 1.25 1.21 1.48 1.27 1.30 

Ovs .\ .87 1.04 .75 1.32 .«) .53 .84 

A vs P .<>1 1.88 M) M M M2 M 



Note. 0 = Opiiinuni or 6 tables 

P = Preaeni ofxrational lablr 

\ = Average of 6 lables from present study 



It should be noted that the values for RMS exceed those for AAD, indicating that a few relatively large errors 
(four percentile points for one raw score in AFQT) e.xist. Inspection of the tables indicates that these deviations are 
generally limited lo very low score ranges. This is probably attribuUble to guessing answers to the test items. 

Table 10 shows the deviations across the five mental category boundary lines for Mie 15,115 subjects in the 
study. The comparison in Tabic 10 is between the conversion table put into effect 1 October 1980 and the form- 
specific tables developed in the present study (six tables in all). Off-diagonal entries are deviations. 

Table 10. Classification by Mental Category Based on 
One Versus Six Tables 



Gitegory by Oper«tional Table 



Category by 

Six Tableii V IV III 



V 934 

IV 177 / 5^15 

III / /l21 5199 224 

n / i 3045 156 

I / I 244 



The proportion of deviations crossing boundaries can be computed by dividing the sum of the off diagonals by 
the sum of all the entries; it is 4.5%. In order to evaluate this percentage, a similar computation was done on the 8a 
form alone (not shown). The comparison was between the operational table outcomes and those from the specific 
Ubie for 8a from the current study. The number of deviations across category lines was 2.4%. This value is useful 
as it presents an estimate of the expected deviations. Clearly the 4.5% representing the comparison of the present 
table versus the six tables is relatively small. 



15 lo 



It was also deemed appropriate to investigate the number of deviations which were 1, 2, 3, or more percentiles 
in magnitude. Table 1 1 shows the deviations crossing categories. As may be observed, most of the deviations are not 
greater than one percentile point. Relatively few ever assume the magnitude of three percentile points and none 
are gnater. It should be noted that for 11,137 subjects no deviations were observed. 





Table } L Deviation of Perce n 


tile Scores across 


Category Lines 










Si/.o of DevialioTis 






\ 


1 point 


2 poini 


3 point 


IV-V 


177 


71% 


29% 




m-iv 


121 


69% 


21% 




ii-iii 


221 


75% 


25% 




III 


ir)6 


35% 


43% 


22% 



IV. CONCLUSIONS 

Forms 8. 9, and 10 of ASVAB were found to be parallel when equated to AFQT-7a, and a single conversion 
table wa>. deenu'd appropriate for operational enlistment processing. 
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